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Abstract Body 



Background/context: 

Increasingly, school-based intervention and whole school reform efforts aim at 
identifying, assessing, and effecting changes in classroom-level processes associated with or 
predictive of children’s social-emotional and academic development (Hamre & Pianta, 2005; 
Pianta, 2006; Raver et al., 2008; Rimm-Kaufman, LaParo, Downer, & Pianta, 2005). These 
classroom-processes are fundamentally social in nature, reflect the underlying quality of the 
interactions among and between teachers and students and encompass emotional, instructional, 
and organizational dimensions of classroom experience. Indeed, in the absence of improving 
such social processes, other resources such as qualified teachers or costly curricular materials 
may be ineffective in promoting learning and achievement (Cohen, Raudenbush & Ball, 2003; 
Fullan, 2001). Reliable and valid methods and measures for assessing important classroom-level 
social processes have recently been developed and are gaining use in the research community 
(Pianta, LaParo & Hamre, 2008). But little is currently known about the ability of school and 
classroom-based interventions to successfully alter these dimensions of classroom settings, 
particularly in poorly functioning classrooms. 

Another important set of constructs that has been neglected in research examining the 
effects of social-emotional learning and literacy development programs may be considered 
together in the domain of teacher affective and pedagogical processes and practices. A teacher’s 
orientation toward their own professional development (Selman, 2003), their perceptions of their 
role in attending to students’ social-emotional needs (Ryan, Gheen, & Midgely, 1998), their 
experience of stress and feelings of job-bumout (Maslach, Jackson & Schwab, 1996; Yoon, 
2002), their classroom management styles and strategies (Wentzel, 2002), and their perceptions 
of their own emotional abilities have each been identified as critical dimensions of teachers 
associated with the development of children’s social and/or academic competence. Teachers’ 
values, beliefs, and perceptions of ability for promoting students’ social-emotional learning may 
also change through their involvement in intervention programs (Adalbjarnardottir, 1994). A 
recent literature review concluded (Daniels & Shumow, 2003) there is a critical need for more 
research on teachers’ views of their role in addressing children’s social-emotional needs. 

Whether teachers feel stressed and overwhelmed, believe in the importance of the intervention 
goals, and feel that they have relevant skills and knowledge all have direct implications both for 
the type of classroom setting they create for children and for the effectiveness of classroom- and 
school-based interventions. 

Purpose / objective / research question / focus of study: 

This presentation capitalizes on a three-year, longitudinal, school-randomized trial of the 
4Rs Program, a comprehensive, school-based social-emotional and literacy program for 
elementary schools, to test intervention induced changes in features of classroom climate and key 
dimensions of teacher affective and pedagogical processes and practices thought to influence 
children’s social-emotional and academic development. Experimental evidence to date suggests 
the 4Rs Program is effective in improving the quality of classroom climate after one-year of 
intervention (Brown, Jones, LaRusso & Aber, accepted pending minor revisions) and positively 
altering the course of children’s social-cognitive, emotional, behavioral and academic 
functioning after two years of intervention (Jones, Brown, & Aber, revision under review) [see 
Findings/Results section below for more detail]. While these results are encouraging, this 
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presentation will extend this work by testing the causal influence of the 4Rs Program on (1) the 
quality of classroom climate at the end of the second and third years of intervention, and (2) 
changes in teachers affective and pedagogical processes and practices over three consecutive 
years of intervention. 

The role of classroom settings and teachers is highlighted in the intervention’s theory of 
change, with the belief that more supportive classroom environments and positive changes in key 
teacher processes and outcomes underlies positive change in students. The 4Rs Program makes a 
significant early investment in teacher preparation beginning with 25 hours of high quality 
teacher training and ongoing professional support throughout the school year. The training 
process encourages teachers to utilize the ideas and skills of the 4Rs curriculum in their own 
lives. The philosophy of the program revolves around the idea that teachers who more strongly 
align their personal beliefs with the values promoted by the program will have classrooms 
where more positive child outcomes occur. As such, the current report on teacher outcomes will 
begin to illuminate the causal process by which the program has yielded positive results for 
children. 

Setting: 

This presentation is based on research that took place between Fall 2004 and Spring 2007 
in eighteen New York City public elementary schools that agreed to participate in a three-year, 
school-randomized study to test the impacts of the 4Rs Program on children, teachers and 
classrooms. School were located across four of the five boroughs of New York City and were 
demographic ally representative of the larger NYC elementary school population. Participating 
schools were pairwise matched and then randomly assigned (see Research Design section below) 
to intervention (9 schools) and control (9 schools) conditions. The 4Rs Program was 
implemented in the intervention schools for three consecutive years. 

Population / Participants / Subjects: 

This study will examine the impact of the 4Rs Program on independent observations of 
classroom climate collected in 147 3 rd and 4 th grade classrooms during the 2005-2006 school 
year and 191 3 rd , 4 th , and 5 th grade classrooms during the 2006-2007 school year. This study will 
also examine the impact of the 4Rs Program on teacher processes and practices, and will include 
299 teachers across the 3 rd , 4 th and 5 th grades. The vast majority of teachers are female (89%), 
and slightly more than half (53%) self-reported as non-Hispanic White, 24% as Black or 
African-American, 15% as Hispanic, and the remaining teachers as Asian, American-Indian, or 
Alaskan Native. Teachers averaged 34 years of age (SD=10 years), 8 years of teaching 
experience overall (SD=6.9) and 5 years at their current school (SD=5). Sixty-eight per cent of 
teachers reported having a Master’s degree, and 30% reported a Bachelor’s degree. Most 
teachers (67%) reported having a New York State Teaching Certificate. 

Intervention / Program / Practice: 

The 4Rs Program is a school-based intervention in literacy development, conflict 
resolution, and intergroup understanding that trains and supports all teachers in grades K-5 in 
how to integrate the teaching of social and emotional skills into the language arts curriculum. It 
is considered a universal intervention in that it targets and is implemented with the entire teacher 
and student population of a given school (Institute of Medicine, 1994). Through the program, 
teachers learn how to use high quality children’s literature as a springboard for helping students 
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gain skills and understanding in the areas of handling anger, listening, assertiveness, cooperation, 
negotiation, mediation, building community, celebrating differences, and countering bias. The 
4Rs Program provides a pedagogical link between the teaching of conflict resolution and the 
teaching of fundamental academic skills, thereby capitalizing on their mutual influence on 
successful youth development (Hinshaw, 1992; Jones et al, revision under review). 

The 4Rs Program has two primary components: (1) a comprehensive 7-unit, 21-35 
lesson, literacy-based curriculum in conflict resolution and social-emotional learning and (2) 25 
hours of training followed by ongoing coaching of teachers to support them in teaching the 4Rs 
curriculum with a minimum of 12 contacts in one school year. 

At its core, the program’s theory of change involves helping teachers more deeply 
assimilate, find utility in, and become skilled at practicing the concepts of the 4Rs Program in 
their own lives and teaching them in their classroom through the consistent delivery of lessons 
from the 4Rs curriculum and the provision of greater social-emotional learning opportunities in 
which students can practice the component skills and be supported in applying them in real life 
situations. The alignment of teachers’ own values, beliefs, and perceptions of ability with the 
underlying pedagogy of a particular intervention is critical to their understanding, acceptance, 
and implementation of the intervention, and to the effectiveness of the intervention itself 
(CPPRG, 1999; Fullan & Stiegelbauer, 1991; Hauer, 2003). Teachers who practice good 
listening skills (e.g., direct eye contact, paraphrasing, acknowledging comprehension) during 
interactions with their students and other adults, and who can teach these skills and provide real- 
life, real-time examples of how they are effective, increase the likelihood their students will 
employ them in their own interactions. But it is not merely the practice of good listening skills by 
the teacher or any given student that is important; it is how the use of these skills reflects a set of 
transactional social processes in the classroom that enable teachers and students to develop 
closer, more intimate relationships (Pianta, 2006; Tseng & Seidman, 2007). Increases in the 
quality of the relationships among teachers and students in turn facilitates future positive 
communications by fostering a more responsive classroom overall. Therefore, central to the 
program’s theory of change is that teachers are successfully engaged in serving as the gateway to 
changing broad characteristics of classrooms including relationships and climate, as well as in 
the development of individual children. 

School-wide implementation of the two primary components of the 4Rs Program 
(curriculum delivery and teacher training and coaching) was systematically tracked and 
monitored during the course of the study. According to implementation data from Year 1, 
teachers in the 9 treatment schools received (a) on average 2.4 (SD=.33) days of training in the 
delivery of the 4Rs curriculum, and (b) an average of 38 (SD=9.6) days per school. On average, 
teachers delivered three-quarters of a lesson per week, with the majority closer to the benchmark 
of 1 lesson per week. The majority of teachers appear to have spent on average between 20-25 
(~40 minutes/week) total hours during Year 1 on 4Rs. Year 2 implementation data revealed a 
slight decrease in training days, and a slight increase in coaching days and the average classroom 
lessons per week, and the amount of time spent on 4Rs per week. Our data also indicate that 
teachers who were trained in the first year of the study, and who remained in the school the 
following year, were even closer to program benchmarks (i.e., on average they implemented 1 
lesson/week and spent ~50 minutes on 4Rs per week). While there is variability in 4Rs 
implementation between teachers and schools, this variation is not inconsistent with similar 
programs and evaluation studies that focus on public schools (e.g., Kam, Greenberg & Walls, 
2003). 
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Research Design: 

This is an experimental (school-randomized), longitudinal study of the 4Rs Program. An 
initial pool of 24 recommended public elementary schools were matched into 12 pairs on 20 
different school-level characteristics (e.g., school size, percent of students receiving free lunch, 
racial/ethnic composition, student attendance and achievement, average spending per student, 
teacher experience) drawn from the 2001-2002 administrative database maintained by the New 
York City Department of Education. A uniform random numbers generator was employed to 
generate, in sequence, 12 random numbers ranging from 0-1 that were assigned to the first 
school in each of the 12 pairs (note, a total of 24 schools were recruited to participate in this 
study and were matched into 12 pairs, the 9 best matching pairs were kept as study schools and 3 
pairs were kept as back-ups). The first school in each pair was assigned to the intervention or 
control group based on the randomly generated number, and the second school in the pair was, 
therefore, assigned to the other group. After random assignment, the two groups were compared 
across the 20 demographic characteristics employed in the matching procedures. As expected the 
two groups did not differ significantly on any of these characteristics and eta2 values (the 
proportion of variance in the demographic characteristic explained by differences between the 
two groups) were minimal. Based on these statistics, the schools can be described as racially and 
ethnically diverse, composed primarily of students who receive a free school lunch, and 
characterized by attendance rates over 89% and one-year stability rates that range from 86% to 
95%. 

Data Collection and Analysis: 

While the overall study involved the collection of data from children, parents, teachers, 
and via independent classroom observations, the focus of this presentation is limited to 
independent observations of classroom climate and teacher self-report data collected from 3 rd 
through 5 th grade classrooms/teachers over three consecutive school years. 

Classroom Climate was assessed via independent observation using the Classroom 
Assessment Scoring System (Pianta, LaParo, & Hamre, 2008) in 3 rd grade classrooms during the 
spring of the first year of the study (n=82), in 3 rd and 4 th grade classrooms during the fall and 
spring of the second year (n=147), and in 3 rd , 4 th and 5 th grade classrooms during the fall and 
spring of the third year (n=191). Observations were conducted by a multiracial/multiethnic field 
research team who received extensive training to reliability standards in the use of this 
instrument. Classrooms were observed for two hours, divided into four 20-minute periods of 
observation, each observation period followed by a 10-minute rating period. CLASS assesses 
three primary domains of classroom climate in preschool through fifth grade classrooms: 
Emotional Support, Classroom Organization, and Instructional Support. Each broad domain is 
comprised of several specific dimensions of interactions. Each dimension, in turn, is represented 
by a continuum of indicators of that dimension, each indicator including a behaviorally 
anchored, observable description of interactions in the classroom (teacher-student, student- 
student). Internal reliability was .90 for Emotional Support, .83 for Classroom Organization, and 
.90 for Instructional Support and .93 for a composite index of all three subscales There is strong 
support for the psychometric properties of the CLASS, including demonstrated relationships to 
children’s social and academic development (predictive validity) both during the preschool (e.g., 
Howes et al., 2008) and elementary school years (e.g., Pianta et al., 2008). 
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Assessments of teacher affective and pedagogical processes and practices were collected 
directly from teachers through paper and pencil survey questionnaires. Similar to the structure of 
classroom observations, teacher surveys were collected during the fall and spring from 3 rd grade 
teachers during the first year of the study, from 3 rd and 4 th grade teachers during the second year 
of the study, and from 3 rd , 4 th , and 5 th grade teachers during the third year of the study. Specific 
measures include: Teaching strategies (Teacher Strategies Questionnaire; Webster-Stratton, 
Reid, & Hammond, 2001), assessing both the Frequency and Usefulness of Positive Strategies 
(12 items, alphas - .76-. 82) and Inappropriate Strategies (9 items each, alphas=.69-.77); Teacher 
Perceptions of Role in Students’ Social-Emotional Well-Being (Ryan, Gheen, & Midgely, 1998), 
a 7-item measure assessing the degree to which teachers believe they play a central role in their 
student’s social-emotional development (alpha=.76-.83); Teacher Burnout (Maslach Burnout 
Inventory-Educators Survey, Maslach, Jackson, & Schwab, 1996), including and overall burnout 
index (18 items, alphs=.88-.91) and subscales of Emotional Exhaustion (9 items, alphas=.88- 
.91), Depersonalization (5 items, alphas=.73-.81), and Personal Accomplishment (7 items, 
alphas=.82-.85); and Perceived Emotioned Intelligence Scale (Brackett & Mayer, 2003), a 17- 
item measure assessing teacher’s self-reported ability to perceive and understand the emotions of 
others and regulate their own emotions (alphas=.72-.87). 

All analyses will be conducted using multi-level hierarchical linear modeling (HLM 
6.02). Variation in classroom climate at the end of Year 2 (Spring 2006, after two years of 
intervention) and the end of Year 3 (Spring 2007, after three years of intervention) will be 
estimated as a function of teacher and classroom demographic control variables at level- 1 (e.g., 
teacher experience, class size) and school factors at level-2 (including treatment status and eight 
dummy variables representing school matched-pair status). Estimates of intervention impact on 
change in the primary teacher processes and practices outcomes from pre-intervention baseline 
(Fall 2004) to the final assessment (Spring 2007) will be calculated using a series of 3-level 
hierarchical linear growth models with school fixed effects. In these models, Level 1 represents 
time (i.e., the 6 repeated assessments of the constructs of interest), Level 2 represents the teacher 
outcomes, and Level 3 represents schools (including treatment status and eight school pair 
dummies). Key teacher- and classroom-level covariates will be included at Level 2. 

Findings / Results: 

Recent findings have demonstrated that the 4Rs Program positively influences both 
classroom-level outcomes. Brown et al (accepted pending minor revisions) found that after one 
year of intervention, classrooms in 4Rs schools were rated by independent observers as higher in 
overall quality compared to classrooms in control schools. Specifically, this work indicated that 
4Rs classrooms had higher mean levels of classroom emotional support and instructional support 
(but not classroom organization) than control classrooms at the end of the first year of the 
intervention. Results of the analyses outlined in the previous section, and currently underway, 
will focus on the impacts of the 4Rs Program on classroom climate in Years 2 and 3 and on 
teacher processes and practices across all three years of the intervention. 

Conclusions: 

Based on these compelling experimental results to date, it is critical to test the impact of 
the 4Rs Program on classroom- and teacher-level outcomes given their centrality as levers for 
student-level change in the 4Rs Program and in a majority of school-based social-emotional 
learning interventions. 
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